Consistency of surrogate risk minimization methods for multiclass 0 - 1 classification
نویسندگان
چکیده
Binary classification Multiclass classification Label space, Y and {±1} [n] Prediction space, T Target 0-1 loss `0-1 : {±1} × {±1} 7→ R+ `0-1 : [n]× [n] 7→ R+ `0-1 : Y × T 7→ R+ `0-1(y, t) = 1(t 6= y) `0-1(y, t) = 1(t 6= y) Surrogate loss ψ : {±1} × C 7→ R+ ψ : [n]× C 7→ R+ ψ : Y × C 7→ R+ where C ⊆ R where C ⊆ R ‘pred’ function pred : C 7→ {±1} pred : C 7→ [n] pred : C 7→ T pred(α) = sign(α) pred(u) = arg max y∈[n] uy
منابع مشابه
Cost-sensitive Multiclass Classification Risk Bounds
A commonly used approach to multiclass classification is to replace the 0− 1 loss with a convex surrogate so as to make empirical risk minimization computationally tractable. Previous work has uncovered sufficient and necessary conditions for the consistency of the resulting procedures. In this paper, we strengthen these results by showing how the 0− 1 excess loss of a predictor can be upper bo...
متن کاملAdversarial Multiclass Classification: A Risk Minimization Perspective
Recently proposed adversarial classification methods have shown promising results for cost sensitive and multivariate losses. In contrast with empirical risk minimization (ERM) methods, which use convex surrogate losses to approximate the desired non-convex target loss function, adversarial methods minimize non-convex losses by treating the properties of the training data as being uncertain and...
متن کاملConsistency of Multiclass Empirical Risk Minimization Methods Based on Convex Loss
The consistency of classification algorithm plays a central role in statistical learning theory. A consistent algorithm guarantees us that taking more samples essentially suffices to roughly reconstruct the unknown distribution. We consider the consistency of ERM scheme over classes of combinations of very simple rules (base classifiers) in multiclass classification. Our approach is, under some...
متن کاملOn the Consistency of Output Code Based Learning Algorithms for Multiclass Learning Problems
A popular approach to solving multiclass learning problems is to reduce them to a set of binary classification problems through some output code matrix: the widely used one-vs-all and all-pairs methods, and the error-correcting output code methods of Dietterich and Bakiri (1995), can all be viewed as special cases of this approach. In this paper, we consider the question of statistical consiste...
متن کاملDesign and Analysis of Consistent Algorithms for Multiclass Learning Problems
We consider the broad framework of supervised learning, where one gets examples of objects together with some labels (such as tissue samples labeled as cancerous or non-cancerous, or images of handwritten characters labeled with the correct character in a-z), and the goal is to learn a prediction model which given a new object, makes an accurate prediction. The notion of accuracy depends on the...
متن کامل